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PREFACE 


The  Department  of  Defense  (DoD)  is  considering  expanding  its  use  of 
contractors  (outsourcing).  This  documented  briefing  examines  the  recent 
history  of  A-76  public/private  cost  comparisons  in  the  DoD  using  data 
covering  fiscal  years  1978-1994.  A-76  refers  to  the  Office  of  Management 
and  Budget  (OMB)  Circular  A-76,  which  describes  the  process  of 
competition  for  work  between  government  employees  and  contractors.  A 
fuller  understanding  of  this  history  may  provide  insights  as  to  how  future 
outsourcing  endeavors  might  progress  as  well  as  suggest  possible  policy 
changes  to  improve  future  outcomes.  The  hope  is  that  outsourcing  might 
reduce  infrastructure  costs  and/or  increase  military  effectiveness. 

This  research  is  part  of  a  project  entitled  "Options  for  Managing  Future 
Budget  Constraints"  undertaken  for  the  Deputy  to  the  Under  Secretary 
(Policy)  for  Policy  Support  in  the  Office  of  the  Under  Secretary  of  Defense 
for  Policy.  The  research  was  conducted  widim  the  International  Security 
and  Defense  Policy  Center  of  RAND's  National  Defense  Research  Institute 
(NDRI),  a  federally  funded  research  and  development  center  sponsored 
by  the  Office  of  the  Secretary  of  Defense,  the  Joint  Staff,  and  the  defense 
agencies. 

The  work  should  be  of  interest  to  individuals  involved  in  A-76  cost 
comparisons  inside  and  outside  the  Department  of  Defense  as  well  as 
policymakers  interested  in  improving  the  comparison  process. 
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SUMMARY 


As  the  Department  of  Defense  considers  expanding  its  use  of  contractors, 
we  attempted  to  glean  lessons  and  insights  from  the  recent  DoD  history  of 
govemment/contractor  cost  comparisons. 

Previous  studies,  e.g.,  Savas  (1992),  Marcus  (1993),  Uttley  (1993),  Tighe  et 
al.  (1996),  argue  there  is  significant  potential  for  savings  and  other 
improvements  from  competition  and  outsourcing.  However,  we  find  that 
since  1978  there  have  been  approximately  five  A-76  cost  comparisons 
canceled  for  every  eight  completed.^  Further,  those  completed  have 
typically  taken  about  two  years  (and  often  considerably  longer)  to 
complete.  If  the  DoD  is  to  fully  benefit  from  competition,  it  would  be 
beneficial  to  imderstand  and  reduce  cancellations  and  delays  in  A-76  cost 
comparisons. 

Outsourcing  in  the  DoD  has  taken  a  variety  of  approaches.  For  example, 
new  missions,  such  as  maintaining  a  new  weapon  system,  can  be  assigned 
to  contractors  directly  without  a  government/ contractor  cost  comparison. 
Also,  there  is  a  process  called  "direct  conversion"  whereby  fimctions 
employing  ten  or  fewer  civilians  can  be  turned  over  to  contractors  without 
a  govemment/contractor  cost  comparison.  However,  this  briefing  focuses 
exclusively  on  the  subset  of  attempted  and  completed  outsourcing  actions 
that  has  involved  government/ contractor  cost  comparisons  that  occurred 
between  FY 1978  and  FY 1994. 

Government/ contractor  cost  comparisons  follow  the  process  described  in 
Office  of  Management  and  Budget  Circular  A-76.  A  cost  comparison  is 
annoxmced,  a  work  statement  describing  the  performance  required  is 
drafted,  and  a  decision  is  reached  as  to  whether  a  contractor  or 
government  employee  "Most  Efficient  Organization"  (MEO)  is  the 
cheapest  approach. 


BACKGROUND  ON  DOD  COST  COMPARISONS 

When  a  function  at  an  installation  is  selected  for  cost  comparison,  a  record 
is  set  up  in  a  data  file  called  the  Commercial  Activity  Management 
Information  System  (CAMIS).  We  obtained  a  copy  of  the  DoD's  CAMIS 


^"A-76"  refers  to  the  Office  of  Management  and  Budget  Circular  that  describes  the 
process  of  competition  for  work  between  government  employees  and  outside  contractors. 
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data  for  FY 1978-1994.  There  were  cost  comparisons  in  the  DoD  prior  to 
1978  and  there  have  been  cost  comparisons  since  1994,  but  our  data  do  not 
cover  them.  The  number  of  cost  comparisons  undertaken  DoD- wide 
dropped  sharply  in  the  1990s,  down  from  a  peak  of  activity  in  the  late 
1970s  and  early  1980s. 

Cost  comparison  cancellation  has  been  a  major  problem.  Whereas  the 
DoD  completed  2268  cost  comparisons  between  FY  1978  and  FY  1994, 
another  1418  were  canceled. 


PATTERNS  OF  COST  COMPARISON  CANCELLATION 

Controlling  for  other  factors,  cost  comparisons  evaluating  large  numbers 
of  government-employed  civilians  were  disproportionately  prone  to 
cancellation.  Indeed,  five  of  the  six  largest  cost  comparisons  (in  terms  of 
the  number  of  government-employed  civilian  slots  evaluated)  attempted 
in  the  DoD  between  FY  1978  and  FY  1994  were  ultimately  canceled.  If  a 
study  is  canceled,  the  status  quo  (generally  provision  of  work  by 
government  employees)  continues. 

The  Marine  Corps,  the  Defense  Logistics  Agency,  and,  especially,  the 
Defense  Mapping  Agency  have  had  problems  with  cost  comparison 
cancellations.  Education  and  Training  and  Manufacturing  cost 
comparisons  have  also  been  vulnerable  to  cancellation. 

Some  cost  comparison  cancellations  may  have  emanated  from  the  policy 
first  promulgated  in  the  FY  1991  Appropriations  Act  of  canceling  single 
function  cost  comparisons  after  two  years  if  they  have  not  yet  reached  bid 
opening.  This  policy  applied  to  cost  comparisons  in  progress  at  the  time 
as  well  as  those  started  since  FY  1991.  Perhaps  reflecting  the  influence  of 
this  policy,  cost  comparisons  started  in  the  late  1980s  were  particularly 
prone  to  cancellation. 


HISTORICAL  DURATION  OF  COST  COMPARISONS 

We  looked  at  the  elapsed  calendar  days  between  the  annoimcement  of  a 
cost  comparison  and  the  date  of  an  initial  decision  as  to  who  should 
provide  the  service.  The  median  cost  comparison  completion  took  664 
days  to  the  initial  decision;  the  mean  was  810  days.  Over  10  percent  of 
completed  cost  comparisons  took  at  least  four  years  to  the  initial  decision. 

We  imdertook  regression  estimation  in  an  attempt  to  find  correlates  with 
completed  cost  comparison  duration.  The  Marine  Corps'  and  the  Army's 


cost  comparisons  have  been  slower  than  the  other  services',  controlling  for 
olher  factors.  Completion  speed  also  varied  by  the  type  of  fimction 
evaluated. 
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Cancellations  and  Delays  in  Completion 
of  DoD  A-76  Cost  Comparisons 
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This  briefing  presents  DoD-wide  data  on  A-76  cost  comparisons  attempted 
between  Fiscal  Year  (FY)  1978  and  FY 1994  (the  last  year  for  which  DoD-wide  data 
are  available).  A-76  refers  to  the  Office  of  Management  and  Budget  Circular  A-76 
that  prescribes  a  specific  process  for  competition  between  government  employees 
and  contractors  for  workload.  We  use  the  term  "cost  comparison"  in  this  briefing  as 
a  shorthand  for  such  A-76  endeavors. 

This  analysis  of  past  cost  comparisons  may  suggest  what  may  occur  if  the  DoD 
chooses  to  expand  such  cost  comparisons  in  the  future.  Further,  pointing  out  some 
past  challenges  may  suggest  areas  for  possible  policy  changes.  Obviously,  the  DoD 
wishes  that  its  outsourcing  efforts  result  in  maximum  possible  cost  savings  and/or 
military  effectiveness. 

It  is  important  to  note,  however,  that  the  DoD  can  and  has  outsourced 
work  through  processes  other  than  the  A-76  procedure  we  analyze  here. 

For  example,  new  missions,  such  as  maintaining  a  new  weapon  system,  can 
be  assigned  to  contractors  directly  without  a  government  contractor  cost 
comparison.  Also,  in  a  process  called  "direct  conversion,"  functions  employing  ten 
or  fewer  civilians  can  be  turned  over  to  contractors  without  a  government/ 
contractor  cost  comparison.  Hence,  this  briefing  analyzes  a  subset,  albeit  an 
important  one,  of  attempted  and  completed  outsourcing  actions  in  ttie  DoD. 
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We  will  first  provide  some  backgrotmd  data  on  cost  comparisons  attempted  in  the 
DoD  between  fiscal  years  1978  and  1994. 
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We  Obtained  the  DoD  CAMIS  Data 
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•information  on  A>76  Cost  Comparisons  DoD-wide 
•Covers  2268  completed,  1418  canceled,  91 
in-progress  cost  comparisons  from  FY 1978 
to  FY  1994 


RAND 


Cost  comparison  data  are  recorded  in  a  system  called  the  Commercial  Activity 
Management  Information  System  (CAMIS).  A  CAMIS  record  is  set  up  every  time 
a  function  is  nominated  for  A-76  cost  comparison.  A  record  is  then  kept  of  these 
nominated  cost  comparisons,  e.g.,  the  function  and  installation  involved,  the  dates 
of  various  milestones,  and,  ultimately,  the  outcome  (completion  or  cancellation)  of 
the  cost  comparison. 

As  noted,  DoD  can  and  has  used  other  avenues  to  outsource  work.  For  example, 
when  10  or  fewer  civilian  positions  are  evaluated,  a  direct  conversion  can  be  used 
where  the  choice  is  only  among  contractors. 

The  data  analyzed  in  this  report,  however,  cover  only  A-76  public /private  cost 
comparisons. 

Further,  the  DoD  outsourced  positions  prior  to  FY  1978.  For  example,  a  large 
installation  support  contract  was  started  at  Vance  Air  Force  Base  in  the  early 
1960s.  See  Shishko,  Paulson,  and  Perry  (1977).  Similarly,  Los  Angeles  Air  Force 
Base  has  used  contracting  extensively  since  a  number  of  maintenance  and  other 
service  contracts  were  put  in  place  prior  to  the  installation  officially  becoming  an 
Air  Force  Station  in  April  1964.  Our  data,  however,  extend  back  only  to  FY  1978. 
Also,  there  have  been  cost  comparisons  since  1994;  they  are  not  considered  here 
either. 
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The  Number  of  Cost  Comparisons  Started 

and  Completed  Sharply  Declined  in  the  mid-1980s 

. . . . . .  . 


Wi  fn-Progress 
□  Canceled 
B  Completed 


Fiscal  Year  Started 
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This  figure  plots  the  number  of  completed,  canceled,  and  in-progress  cost 
comparisons  by  year  started,  as  of  FY  1994,  the  last  year  for  which  such  data  are 
available  DoD-wide.  We  see  a  sharp  diminution  in  the  number  of  cost 
comparisons  started  from  FY  1985  forward,  with  very  few  comparisons  started 
DoD-wide  in  the  1990s.  Further,  the  figure  shows  the  rate  of  cancellation  was 
greater  for  those  cost  comparisons  started  in  the  late  1980s  rather  than  in  the  early 
1980s.  Hence,  between  the  reduction  in  the  number  of  comparisons  started  and 
the  increase  in  the  comparison  cancellation  rate,  there  was  a  sharp  drop  in  the 
number  of  completed  comparisons. 

In  total,  this  figure  covers  2268  completed,  1418  canceled  (or  roughly  an  8  to  5 
ratio  of  completed-to-canceled  comparisons),  and  91  cost  comparisons  in  progress 
as  of  FY  1994.  Some  of  the  in-progress  designations  seem  somewhat  suspect, 
though.  For  example,  there  are  15  cost  comparisons  annormced  in  FY  1982  still 
officially  recorded  as  m  progress. 
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Government-Employed  Civilian  Slot 
Evaluation  Peaked  in  the  Early  1980s 
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This  figure  shows  the  number  of  government-employed  civilian  slots  evaluated 
by  starting  fiscal  year.  In  aggregate,  70,897  civilian  slots  were  evaluated  in 
completed  cost  comparisons;  another  59,374  were  evaluated  in  cost  comparisons 
that  were  ultimately  canceled. 

On  page  4,  we  noted  there  was  roughly  an  8  to  5  ratio  of  completed  to  canceled 
cost  comparisons.  On  the  basis  of  civilian  slots  evaluated,  however,  the 
completed /canceled  ratio  is  closer  to  6  to  5.  This  result  presages  an  upcoming 
finding  that  cost  comparisons  that  evaluated  large  numbers  of  government- 
employed  civilians  were  disproportionately  prone  to  cancellation. 
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FY  1987  Had  the  Most  Military  Slots 
Successfully  Evaluated 
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Evaluation  of  military  slots  in  A-76  cost  comparisons  peaked  somewhat  later  than 
evaluation  of  civilian  slots. 

Closely  mirroring  page  4's  results,  there  was  roughly  an  8  to  5  ratio  of  military 
slots  in  completed  A-76  cost  comparisons  to  military  slots  in  canceled  cost 
comparisons. 

In  addition  to  the  slots  in  this  figure,  many  military  slots  were  evaluated  in  direct 
conversions.  See  Keating,  Camm,  and  Hanks  (1997). 
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The  Navy  and  Air  Force  Have  Started  and 
Completed  the  Most  Cost  Comparisons 


This  figure  plots  the  number  of  completed  and  canceled  cost  comparisons  by 
service.  The  Navy  has  started  (1337)  and  completed  (883)  the  most  cost 
comparisons.  The  Air  Force's  totals  (1192  started,  767  completed)  are  close  behind 
the  Navy's. 
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Command  Installations  Have  Different 
Cost  Comparison  Histories 


Installation  Command 


O  Canceled 
M  Completed 


RAND 


This  figure  reprises  page  7's  data,  but  breaks  the  Air  Force  and  Army  into 
constituent  command  installations  and  the  defense  agencies  into  the  three  DoD 
agencies  that  have  undertaken  cost  comparisons.  The  number  of  comparisons 
started  and  completed  varies  considerably  across  the  commands. 

Defining  the  acronyms:  ACC  is  Air  Combat  Command,  AETC  is  Air  Education 
and  Training  Command,  AFAMC  is  Air  Force  Air  Mobility  Command,  AFMC  is 
Air  Force  Materiel  Command,  AFSPC  is  Air  Force  Space  Command,  PACAF  is 
Pacific  Air  Forces,  AFOth  is  other  Air  Force  commands  (e.g..  Special  Operations 
Command),  ArMC  is  Army  Materiel  Command,  FORS  is  Forces  Command, 
TRAD  is  Training  and  Doctrine  Command,  ArOth  is  other  Army  commands  (e.g.. 
United  States  Army  Pacific),  DeCA  is  Defense  Commissary  Agency,  DLA  is 
Defense  Logistics  Agency,  and  DMA  is  Defense  Mapping  Agency.  For  the  Army 
and  Air  Force,  the  installations  are  identified  with  their  current  conunands.  It  is 
possible  the  installations  were  actually  associated  with  different  commands  (for 
example,  the  now-extinct  Strategic  Air  Command)  when  the  cost  comparison  was 
undertaken. 
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Next,  we  are  going  to  analyze  the  pattern  of  cost  comparisons  canceled  versus 
completed.  Because  there  have  been  roughly  five  cost  comparisons  canceled  for 
every  eight  completed,  and  considering  the  nontrivial  costs  of  fruitlessly 
imdertaking  a  cost  comparison,  cost  comparison  cancellation  has  been  a  serious 
problem. 
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Defense  Agency,  Marine  Corps  Cost  Comparisons 
Have  Been  Most  Frequently  Canceled 


We  undertook  a  probit  estimation  to  evaluate  whether  service,  type  of 
function,  time  period,  number  of  slots  evaluated,  or  other  factors  influenced 
the  probability  a  cost  comparison  would  be  completed  versus  canceled.  (In- 
progress  cost  comparisons  were  not  considered.)  The  appendix  details  the 
probit  estimation  results. 

In  this  figure,  we  show  our  probit  estimates  for  the  propensity  of  services 
to  complete  their  cost  comparisons,  controlling  for  differences  in  the  type 
of  function,  time  period,  number  of  slots  evaluated,  and  other  factors. 

The  middle  tick  on  each  line  represents  our  best  probit  estimate  of  the 
services'  probability  of  completing  their  cost  comparisons  with  other 
variables  evaluated  at  their  mean  values.  The  line  on  each  side  of  the  tick 
approximates  plus  and  minus  one  standard  error  around  each 
service's  estimate.  Two  services'  estimates  are  statistically  significantly 
different  at  the  95  percent  confidence  level  if  their  lines  do  not  overlap. 

This  estimation  suggests  the  Defense  agencies  and  the  Marine  Corps  (USMC) 
have  had  particular  difficulty  in  concluding  cost  comparisons,  controlling  for 
other  factors.  The  Navy  has  had  the  greatest  success  in  completing  cost 
comparisons.  There  could  be  lessons  from  the  three  largest  services  for 
the  USMC  and  the  defense  agencies. 


DMA,  DLA,  Marine  Corps  Cost  Comparisons  Have 
Been  Most  Frequently  Canceled 


RAND 


This  figure  repeats  page  lO's  analysis,  but  breaks  the  Army,  Air  Force,  and  DoD 
agencies  into  constituent  components.  Among  Air  Force  command 
installations.  Pacific  Air  Force  (PACAF)  installations  have  had  particular 
problems  in  completing  cost  comparisons.  Army  Materiel  Command  (ArMC) 
installations  have  been  most  successful  in  completing  cost  comparisons. 
Meanwhile,  among  DoD  agencies,  the  Defense  Commissary  Agency  (DeCA) 
has  done  comparatively  well,  but  the  Defense  Logistics  Agency  (DLA)  and, 
especially,  the  Defense  Mapping  Agency  (DMA)  have  had  problems  completing 
cost  comparisons. 
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Manufacturing,  Education  and  Training  Cost 
Comparisons  Have  Been  the  Most  Likeiy  to  Be  Cancelled 

. .  .  . 


RAND 


This  figure  shows  the  estimated  probabilities  of  concluding  cost  comparisons 
by  function,  controlling  for  other  factors.  Base  Maintenance  (BaseMain), 
Intermediate  Maintenance  (IMaint),  and  Installation  Services  (Install)  cost 
comparisons  were  most  often  concluded.  Cost  comparison  cancellation  has 
been  most  common  for  In-House  Manufacturing  (Manuf)  and  Education  and 
Training  (Educat). 

Defining  the  rest  of  the  abbreviations:  ADP  is  Automatic  Data  Processing, 
Depot  is  Depot  Repair,  Health  is  Health  Services,  OtherNon  is  Other 
Nonmanufacturing,  R&D  Support  is  Research  and  Development  support, 
RPM  is  Real  Property  Maintenance,  and  Social  is  Social  Services.  Unlike 
R&D  support,  pure  Research  and  Development  is  exempted  from  A-76 
cost  comparison.  In  practice,  however,  this  distinction  is  fuzzy.  See  Tighe, 
Trimkey,  and  Kleinman  (1996). 
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Late  1980s’  Cost  Comparisons  Were 
Particularly  Vulnerable  to  Cancellation 
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This  figure  shows  that  cost  comparisons  started  in  the  late  1980s  were 
particularly  vulnerable  to  cancellation.  One  possible  reason  is  the  policy 
first  promulgated  in  the  FY  1991  Appropriations  Act  of  canceling  single 
function  initiatives  after  two  years  and  multifunction  initiatives  after 
four  years  if  they  have  not  yet  reached  bid  opening.  This  legislation 
applied  to  comparisons  in  progress  at  the  time,  as  well  as  those  started 
FY  1991  and  later.  Hence,  a  comparison  started  in  FY  1987  but  not  yet 
completed  in  FY  1991  was  to  be  immediately  terminated. 

It  is  also  interesting  to  recall  that  page  4  indicated  the  number  of 
comparisons  started  plunged  in  the  late  1980s.  An  overburdening 
of  contracting  personnel  was  clearly  not  an  explanation  for  this 
figure's  pattern. 
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The  More  Government-Employed  Civilians  Evaluated, 
the  More  Likely  Is  Cost  Comparison  Canceilation 


Civilian  Slots  Evaluated 


RAND 


Our  estimation  suggests  that  cost  comparisons  involving  more  government- 
employed  civilians  were  less  likely  to  be  completed,  controlling  for  other  factors. 
The  center  line  in  the  above  figure  is  our  best  probit  estimate  of  this  tendency; 
the  other  lines  are  plus  and  minus  two  standard  errors  aroimd  this  estimate.  The 
data  suggest  large  cost  comparisons  have  been  difficult  to  complete. 

The  scale  on  this  figure  was  constructed  to  cover  the  range  of  cost 
comparison  sizes  attempted  DoD-wide.  The  median  attempted  cost 
comparison,  however,  evaluated  only  12  civilian  slots  whereas  only  5  percent  of 
the  attempted  comparisons  evaluated  more  than  150  civilian  slots.  As  discussed 
in  file  appendix,  we  used  the  square  root  of  the  number  of  civilian  slots 
evaluated  as  the  independent  variable  to  reduce  the  right 
skewness  in  these  data. 

The  largest  competition  started  in  the  data  set  was  a  1407  civilian  slot 
clothing  and  textiles  cost  comparison  at  the  Defense  Personnel  Support 
Center  (DPSC)  in  Philadelphia  that  was  canceled.  The  second  largest 
attempted  cost  comparison  and  the  largest  completed  cost  comparison 
was  a  1057  slot  installation  support  competition  at  Aberdeen  Proving 
Groimd.  However,  the  3rd  through  6th  largest  attempted  cost  comparisons  (853 
slots  at  the  Fort  Benning  Directorate  of  Engineering  and  Housing,  840  slots 
at  the  Fort  Benning  Directorate  of  Logistics,  811  slots  in  base  supply 
operations  at  Sunny  Point  Military  Ocean  Terminal,  and  790  slots  at  the 
Crane  Army  Ammimition  Plant  Directorate  of  Logistics)  were  all  canceled. 
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The  appendix  discusses  a  supplemental  estimation  that  suggests  these 
civilian  slot  cancellation  problems  are  most  acute  in  the  Army.  By  contrast, 
the  larger  Air  Force  cost  comparisons  were  not  disproportionately  prone  to 
cancellation. 

Bolten,  Halliday,  and  Keating  (1996)  note  that  the  Army's  Fort  Riley 
Directorate  of  Logistics  has  chosen  to  have  a  set  of  small  contracts. 

By  contrast,  the  Fort  Riley  Directorate  of  Public  Works  attempted  to  set  up 
a  large  contract,  but  failed  to  do  so  after  six  years  of  effort.  These  results 
are  consistent  with  the  Fort  Riley  experience. 
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Next,  we  provide  some  backgroxmd  information  on  those  cost  comparisons 
that  were  completed. 
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Cost  Comparison  Completions  Peaked 
in  the  Early  1980s 


RAND 


The  above  figure  tallies  completed  cost  comparisons  by  year  of  initial  decision. 
Fiscal  Year  (FY)  1983  with  370  completions  was  the  DoD's  peak.  By  contrast, 
only  113  cost  comparisons  were  completed  in  the  first  five  fiscal  years  of  the 
1990s. 

In  this  and  subsequent  figures,  we  look  at  2134  cost  comparison  completions, 
rather  than  the  2268  discussed  earlier.  We  limited  ourselves  to  cost 
comparisons  for  which  the  C AMIS  record  indicates  the  dollar  value  and 
duration  of  the  award.  Completed  cost  comparisons  missing  these  data  were 
not  useful  in  our  analysis. 
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Government  Employees,  Contractors  Each 
Won  About  Half  of  Cost  Comparisons 


RAND 


Government  employees  (here  labelled  "MEO"  for  Most  Efficient  Organization,  the 
designation  given  to  the  government  employees'  proposed  approach)  and 
contractors  have  almost  exactly  split  the  2134  completed  cost  comparisons. 
Government  MEOs  have  won  1055  cost  comparisons  versus  1079  won  by 
contractors. 
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Contractors  Have  Had  the  Greatest 
Success  Rate  in  the  Air  Force 
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The  above  figure  tallies  the  percentage  of  cost  comparisons  won  by  contractors 
versus  government-employee  MEOs  across  the  services.  Contractors  have  had 
greatest  success  in  the  Air  Force,  winning  60.0  percent  of  cost  comparisons,  versus 
47.5  percent  in  the  Army  and  43.6  percent  in  the  Navy. 
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Contractor  Success  Rates  Vary  Widely 
Across  Commands  and  Agencies 


RAND 


The  percentage  of  contractor  versus  MEO  awards  across  Army  and  Air  Force 
command  installations  plus  the  defense  agencies  are  shown  above.  In  percentage 
terms,  contractors  have  been  least  successful  in  the  Defense  Logistics  Agency 
(winning  7.1  percent  of  cost  comparisons)  and  at  Army  Materiel  Command 
installations  (39.1  percent)  and  most  successful  in  the  Defense  Commissary 
Agency  (71.8  percent)  and  at  Air  Force  Space  Command  installations  (70.5 
percent). 
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Real  Dollar  Value  of  Completed  Cost 
Comparisons  Has  Declined  Since  FY  1980 
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This  figure  presents  the  annualized  FY  1994  dollar  value  of  completed  cost 
comparisons  by  year  of  completion. 

The  annualized  value  represents  the  cost  of  the  winning  contractor  or 
government-employee  bid  divided  by  the  agreement's  duration  in  years.  The 
appendix  presents  the  multipliers  used  to  convert  all  totals  into  FY  1994  dollars. 

In  annualized  FY  1994  dollars,  FY  1980  was  the  peak  year  for  completed  cost 
comparisons  at  $557  million.  By  contrast,  $132  million  worth  of  cost  comparisons 
were  completed  in  the  first  five  fiscal  years  of  the  1990s. 

There  is  no  clear  pattern  in  the  average  annualized  FY  1994  value  per  completion 
by  fiscal  year.  The  median  completed  cost  comparison  had  an  annualized  value  of 
$474,000;  the  mean  was  $1,515,000.  Five  percent  of  the  completions  had 
annualized  value  less  than  $59,000;  5  percent  had  annualized  value  greater  than 
$6,076,000.  The  average  value  ranged  from  $0.9  million  for  comparisons 
completed  in  FY  1984  to  $3.9  million  for  the  two  comparisons  completed  in  FY 
1993. 
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Contractors  Won  Somewhat  More  Dollar 
Value  Than  Government  Employees 
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In  annualized  FY  1994  dollar  terms,  contractors  have  won  about  $1.8  billion  worth 
of  cost  comparisons  (57.1  percent)  versus  about  $1.4  billion  won  by  government- 
employee  MEOs  (42.9  percent).  Hence,  combining  this  figure's  results  with  page 
18  s,  it  follows  that  the  average  cost  comparison  won  by  contractors  is  somewhat 
larger  than  the  average  cost  comparison  won  by  government  employees  ($1.7 
million  versus  $1.3  million  in  FY  1994  dollars). 
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The  Army  Has  Completed  Cost  Comparisons 
Encompassing  the  Greatest  Value 


Service 


RAND 


This  figure  shows  annualized  dollars  won  by  service.  Note  that  whereas  the  Navy 
and  the  Air  Force  completed  the  most  cost  comparisons  (776  and  760,  respectively, 
versus  501  for  the  Army),  the  Army's  completed  cost  comparisons  sum  to  the 
greatest  annualized  value.  The  Army's  average  completed  cost  comparison  was 
worth  about  $2.5  million  in  armualized  FY 1994  value  versus  $1.3  million  for  the 
Air  Force  and  $1.1  million  for  the  Navy. 
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TRADOC,  ArMC  Installation  Comparisons 
Covered  Greatest  Value 


RAND  I 


The  Army's  Training  and  Doctrine  Command  (TRAD)  installations  and 
Materiel  Command  (ArMC)  installations  had  the  greatest  value  in 
completed  cost  comparisons,  notwithstanding  that  page  8  showed  the 
Army's  Forces  Command  installations  (158)  as  well  as  the  Air  Force's  Air 
Combat  Command  installations  (157),  Air  Education  and  Training 
Command  installations  (140),  and  Air  Force  Materiel  Command  installations 
(149)  completed  more  cost  comparisons  than  either  TRAD  installations  (132) 
or  ArMC  installations  (87). 
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A-76  Cost  Comparisons  Primarily  Evaluated 
Government-Employed  Civilians 
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Most  A-76  cost  comparisons  have  evaluated  government-employed  civilian  slots 
rather  than  military  slots.  In  total,  65,741  government-employed  civilians  and 
18,973  military  had  been  evaluated  in  cost  comparisons  as  of  FY  1994.  The 
median  completed  cost  comparison  evaluated  12  civilian  slots;  the  mean  was  31 
civilian  slots.  Five  percent  of  the  completions  evaluated  1  or  0  civilian  slots;  5 
percent  evaluated  125  or  more  government-employed  civilicin  slots.  The  median 
completed  cost  comparison  evaluated  0  military  slots;  the  mean  was  9  military 
slots.  Five  percent  of  the  completions  evaluated  26  or  more  military  slots. 
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Contractors  Have  Won  80%  of  Military  Slots, 
Half  of  Civilian  Slots  Subject  to  Cost  Comparison 
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Not  surprisingly,  contractors  have  had  a  lower  success  rate  winning  government 
civilian  slots  cost-compared  versus  military  slots  cost-compared.  52.7  percent  of 
government  civilian  slots  subject  to  cost  comparison  resulted  in  a  contractor 
winning,  but  contractors  won  78.8  percent  of  the  military  slots  subject  to  cost 
comparison.  Government  employees  are  much  better  positioned  to  retain 
workload  they  currently  perform  than  to  acquire  worldoad  currently  performed 
by  military  personnel. 

Marcus  (1993)  argues  that  commercial  activities  performed  predominantly  by 
military  personnel  hold  disproportionate  potential  for  cost  savings. 
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In  All  Services,  Contractors  Have 
Usually  Won  Military  Slots 


RAND 


Echoing  page  26,  contractors  won  the  predominant  portion  of  military  slots  cost- 
compared  in  all  services  (e.g.,  91.5  percent  in  the  Navy,  74.8  percent  in  the  Army, 
74.2  percent  in  the  Air  Force).  Contractors  have  had  less  success  winning  civilian 
slots,  although  contractors  have  won  more  than  half  the  civilian  slots  competed  in 
the  Army  (59.4  percent)  and  Air  Force  (56.6  percent). 
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Contractors  Have  Had  Greatest  Success 
in  Air  Force  Cost  Comparisons 
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We  ran  a  probit  estimation  to  analyze  when  contractors,  as  opposed  to 
government-employee  MEOs,  have  won  cost  comparisons.  The  details  are  in  the 
appendix. 

This  estimation  controls  for  the  fact  that  different  services  have,  for  example, 
different  functional  mixes.  Raw  contractor  win  rates,  as  discussed  above,  do  not 
control  for  such  functional  mix  differences. 

Hence,  for  example,  whereas  page  27  showed  contractors  had  a  slightly  higher 
success  rate  competing  for  civilian  slots  in  the  Army  than  in  the  Air  Force,  this 
estimation  suggests  that  contractors  have  had  the  greatest  relative  success  in  Air 
Force  cost  comparisons,  controlling  for  other  factors,  such  as  the  type  of  function 
competed. 
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Contractors  Have  Done  Best  at  Air  Force  Space 
Command  Installations,  Worst  in  DoD  Agencies 


RAND 


This  figure  breaks  up  page  29' s  results  for  the  Air  Force  and  Army  command 
installations. 
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Contractors’  Win  Rates  Have  Varied 
Significantly  by  Function 


RAND 


Contractors  have  had  the  greatest  success  in  Social  Services'  cost  comparisons  and 
the  least  success  in  Health  Services'  cost  comparisons,  controlling  for  other  factors. 

There  were  nine  completed  Depot  Repair  cost  comparisons,  all  won  by 
government-employee  MEOs,  and  two  completed  Manufacturing  cost 
comparisons,  both  won  by  contractors.  Because  of  the  lack  of  variance  in  these 
functions'  outcomes,  they  could  not  be  included  in  this  probit  estimation. 
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Contractors  Had  Least  Success 
in  the  Late  1980s 


RAND 


Controlling  for  other  factors,  contractors  had  least  success  winning  cost 
comparisons  completed  in  the  late  1980s. 

The  1993-1994  standard  error  is  large  because  of  the  small  number  of  comparisons 
completed  during  those  years. 
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Contractors  Have  Had  Greater  Success 
When  Military  Slots  Are  Evaluated 


RAND 


Our  results  suggest  that  cost  comparisons  that  predominately  involve  military- 
held  slots  are  more  likely  to  be  won  by  a  contractor  than  cost  comparisons 
involving  government-employed  civilian-held  slots.  It  is  not  surprising  that 
government  employees  appear  to  be  better  positioned  to  win  cost  comparisons 
involving  functions  already  performed  predominately  by  government-employed 
civilians. 
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Next,  we  analyze  the  durations  of  completed  cost  comparisons. 
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The  Cost  Comparison  Process  Has  Been 
Lengthy  and  Variable 


I  RAND I 


The  DoD  C AMIS  file  contains  date  stamps  that  record  the  duration  of  the  cost 
comparison  process.  The  above  figure  plots  the  distribution  of  realized  durations 
between  the  date  a  cost  comparison  is  announced  and  the  date  of  an  initial 
decision.  We  focus  on  the  time  from  the  armouncement  date  to  the  initial  decision 
date  in  this  analysis.  The  performance  start  date  is  naissing  on  many  records. 
However,  when  it  is  included,  the  winning  contractor  or  govenament  employees 
often  started  performance  prior  to  the  final  decision  date.  The  final  decision  date 
falls,  on  average,  92  days  after  the  initial  decision. 

The  figure  shows  that  the  cost  comparison  process  has  been  both  lengthy  (median 
duration  of  664  days,  mean  duration  of  810  days)  and  highly  variable.  Five 
percent  of  the  completed  cost  comparisons  took  189  or  fewer  days;  5  percent  took 
1902  or  more  days. 
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The  Marine  Corps  and  Army  Cost  Comparisons 
Have  Been  the  Slowest 
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We  attempted  to  find  independent  variables  that  explained  the  realized  duration 
of  cost  comparisons.  We  looked  at  factors  such  as  the  command /service, 
function,  year  of  completion,  and  the  number  of  slots  evaluated.  See  the  appendix 
for  the  complete  results. 

This  figure  shows  estimates  of  typical  duration  by  service  as  well  as  one  Tukey- 
Kramer  adjusted  standard  error  on  each  side  of  the  estimate.  These  estimates 
come  from  an  analysis  of  covariance  that  used  the  square  roots  of  realized 
durations  as  the  dependent  variable.  The  point  estimates  displayed  do  not 
necessarily  correspond  in  magnitude  to  sample  means  in  the  data.  Hence,  though 
the  mean  cost  comparison  ran  810  days,  this  type  of  display  shows  point  estimates 
that  are  generally  larger  than  810  days.  See  the  appendix  plus  Pearce  (1982),  Miller 
(1985),  and  SAS  Institute  (1990b)  for  discussions  of  this  estimation  procedure. 

Controlling  for  other  factors,  the  Marine  Corps  and  Army  completed  cost 
comparisons  have  taken  statistically  significantly  longer  than  the  Air  Force's. 
Combining  this  figure's  results  with  page  lO's,  we  see  a  troubling 
consistency  that  the  Marine  Corps  had  disproportionate  trouble  completing 
cost  comparisons  and  their  completions  were  unusually  slow. 
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Forces  Command  Installations  Have 
Had  the  Slowest  Cost  Comparison  Completions 
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This  figure  provides  more  depth  to  page  36's  findings.  The  Army  Forces 
Command  installations  have  been  the  slowest  at  completing  cost  comparisons, 
controlling  for  other  factors.  The  Air  Force's  Air  Education  and  Training 
Command  installations  and  Space  Command  installations  have  been  fastest. 
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Data  Processing  Cost  Comparisons 
Have  Been  Particulariy  Lengthy 
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The  estimation  suggests  Automatic  Data  Processing  (ADP),  Base  Maintenance, 
and  Health  Services  cost  comparisons  were  particularly  lengthy.  Education  and 
Training  and  In-House  Manufacturing  cost  comparisons  were  completed  most 
rapidly.  Recall,  however,  that  page  12  suggests  these  functions  were  least  likely  to 
have  their  cost  comparisons  completed.  Hence,  their  apparent  rapid  speed  may 
simply  be  a  result  of  data  censoring  from  widespread  cost  comparison 
cancellation. 
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Cost  Comparisons  May  Be 
Getting  More  Rapid 
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This  figure  suggests  that  cost  comparisons  completed  in  recent  years  have  been 
completed  somewhat  more  rapidly.  The  policy  of  canceling  incomplete  smgle 
function  cost  comparisons  after  two  years  may  be  one  reason  for  this  ^ding.  The 
policy  may  be  speeding  initiatives  or,  referring  to  page  13,  perhaps  this  apparent 
speed  increase  is  at  least  in  part  a  data  censoring  caused  by  more  cost 
comparisons  being  canceled. 

Also,  there  have  been  many  fewer  completions  in  recent  years.  Speed  increases 
may  be  a  result  of  fewer  cost  comparisons.  Cost  comparison  personnel  have  had 
less  work  to  do,  so  perhaps  they  have  been  able  to  complete  the  comparisons  they 
have  undertaken  more  quickly.  Alternatively,  perhaps  learning  has  expedited  the 
process. 

The  diminution  in  the  number  of  completions  in  recent  years  also  causes  the 
standard  error  on  the  1993  estimate  to  be  larger  than  the  other  periods'  estimates' 
standard  errors. 

We  omit  early  1980s'  completions  from  this  figure.  Comparisons  completed  in  the 
early  1980s  were  quite  rapid,  but  this  is  somewhat  tautological.  Since  this  data 
set's  cost  comparisons  started  only  in  the  late  1970s,  any  comparisons  completed 
in  the  early  1980s  were,  by  definition,  fairly  rapid. 
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NDRJ 


Outline 


•Background  on  DoD  Cost  Comparisons 
•Patterns  of  Cost  Comparison  Cancellation 
•Background  on  Completed  Cost  Comparisons 
•Historical  Duration  of  Cost  Comparisons 
- ►  •Concluding  Observations 

. .  RAND 

We  provide  a  few  summary  observations  to  conclude. 
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Concluding  Observations 


•  Cost  Comparisons  Have  Been  Lengthy 

•  Cost  Comparison  Cancellation  Has  Been  a  Major 
Problem 

.  Evaluation  of  More  Government-Employed 
Civilians  Appears  to  Increase  the  Probability  of 
Cancellation 

.  Cost  Comparisons  Have  Been  Rare  in  the  1990s 


I  RAND I 


This  briefing  has  made  a  number  of  points.  Completed  A-76  cost  comparisons 
have  taken,  on  average,  aroimd  two  years.  Historically,  about  five  cost 
comparisons  have  been  cancelled  for  every  eight  completed.  It  appears  that  cost 
comparisons  that  evaluated  large  numbers  of  government-employed  civilians 
have  been  particularly  vulnerable  to  cancellation.  There  has  been  a  sharp 
diminution  in  the  number  of  cost  comparisons  attempted  and  completed  in  the 
1990s  after  a  peak  in  the  early  1980s. 

Again,  however,  we  reiterate  our  caveat  that  this  analysis  covers  only 
A-76  cost  comparisons.  There  are  other  approaches  the  DoD  can  and 
has  used  to  increase  the  role  of  contractors  in  providing  goods  and 
services  to  the  military. 
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APPENDIX:  AN  ANALYSIS  OF  THE  DOD 
CAMIS  DATA  SET 


We  obtained  a  copy  of  the  Department  of  Defense  (DoD)  Commercial 
Activities  Management  Information  System  (CAMIS)  data  base.  The  data 
we  obtained  covered  FY 1977-1994. 

The  CAMIS  data  track  and  record  information  about  A-76  goverrunent/ 
contractor  cost  comparisons.  A-76  is  the  Office  of  Management  and 
Budget  circular  describing  the  process  by  which  government  employees 
and  contractors  compete  for  the  opportunity  to  provide  a  service  on  an 
installation.  These  data  do  not  include  other  sorts  of  outsourcing 
initiatives,  such  as  direct  conversions  where  the  only  choice  is  among 
contractors.  Only  A-76  cost  comparisons  are  covered  in  this  analysis. 

A  CAMIS  record  is  set  up  any  time  a  function  is  nominated  for  potential 
outsourcing.  Various  milestones  are  tracked  in  CAMIS.  Data  are 
recorded  on  the  type  of  function  involved,  the  contract  approach  (e.g., 
sealed  bid,  negotiated),  the  winner  of  the  comparison,  the  dollars 
involved,  and  many  other  characteristics  of  the  process. 

Not  all  cost  comparisons  that  are  started  are  completed.  Indeed,  as  shown 
in  Table  A.l,  approximately  five  cost  comparisons  have  been  canceled  for 
every  eight  completed  in  the  DoD.  Table  A.l  also  shows  that  the  number 
of  comparisons  started  has  diminished  sharply  since  a  peak  in  FY  1981. 
Table  A.l  and  subsequent  analyses  ignore  CAMIS  records  that  do  not 
record  the  date  a  cost  comparison  started. 

Table  A.2  shows  the  current  status  of  cost  comparisons  by  service.  The 
Marine  Corps  and  DoD  agencies  (the  Defense  Commissary  Agency,  the 
Defense  Logistics  Agency,  and  the  Defense  Mapping  Agency)  have 
canceled  more  comparisons  than  they  have  completed. 
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Table  A.l 

DoD  Cost  Comparison  Status  by  Fiscal  Year  Started 


Start 

FY 

Completed 

Canceled 

In  Progress 

1977 

1 

0 

0 

1978 

70 

21 

0 

1979 

499 

55 

1 

1980 

65 

7 

0 

1981 

555 

199 

6 

1982 

371 

215 

15 

1983 

319 

225 

6 

1984 

206 

201 

0 

1985 

38 

62 

0 

1986 

36 

83 

0 

1987 

37 

115 

8 

1988 

9 

62 

0 

1989 

20 

142 

16 

1990 

25 

4 

14 

1991 

9 

18 

1 

1992 

7 

7 

5 

1993 

1 

2 

9 

1994 

0 

0 

10 

Total 

2268 

1418 

91 

Table  A.2 

DoD  Cost  Comparison  Status  by  Service 


Service 

Completed 

Canceled 

In  Progress 

Air  Force 

767 

405 

20 

Army 

517 

315 

39 

Navy 

883 

454 

1 

Marine  Corps 

44 

79 

1 

DoD  agencies 

57 

165 

30 

Total 

2268 

1418 

91 

In  light  of  the  frequency  and  obvious  cost  of  cost  comparison  cancellation, 
we  ran  a  probit  estimation  to  see  if  there  has  been  a  pattern  in  the  sorts  of 
comparisons  that  have  been  canceled.  [Probit  estimation  is  akin  to  linear 
regression  but  is  more  appropriate  in  a  case  like  this  with  a  dichotomous 
dependent  variable.  See,  for  example,  Cramer  (1989).]  Table  A.3  lists  the 
independent  variables  used.  All  independent  variables  except  the 
constant  and  the  square  roots  of  the  number  of  military  and  civilian  slots 
evaluated  in  the  cost  comparisons  are  dummy  variables.  We  took  the 
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square  roots  of  the  number  of  military  and  civilian  slots  evaluated  to  make 
these  slot  independent  variables  more  normally  distributed.  These  two 
square  root  slot  terms  are  not  highly  correlated;  their  Pearson  Correlation 
Coefficient  in  these  data  is  0.20323.  Table  A.3  groups  the  dummy 
variables  into  sets.  Footnote  (a)  discusses  the  omitted  variable  for  each  set. 
The  commands  are  those  currently  associated  with  the  installation. 
Installations  may  have  been  in  other  coimnands  when  the  cost  comparison 
was  started.  The  function  (e.g..  Automatic  Data  Processing,  Base 
Maintenance)  is  the  first  fimctional  category  listed  in  the  CAMIS  fimction 
field.  Some  comparisons  involved  more  than  one  fimctional  category. 

This  analysis,  however,  only  uses  the  first  listed  fimctional  category. 

Table  A.4  presents  our  completion  probit  estimates.  The  table  suggests 
increasing  the  number  of  civilian  slots  evaluated  decreases  the  probability 
a  comparison  is  completed,  holding  other  factors  constant.  Government- 
employed  civilians  may  have  been  able  to  impede  cost  comparisons  in  a 
manner  that  decreased  the  probability  of  completion. 

Among  the  commands  and  services,  the  Marine  Corps,  the  Defense 
Logistics  Agency,  and,  especially,  the  Defense  Mapping  Agency  have  had 
particular  problems  completing  comparisons.  Controlling  for  other 
factors.  Army  Materiel  Command  installations  have  had  the  greatest 
success  completing  comparisons. 

There  is  considerable  variability  across  functions  in  cancellation  rates. 
Manufacturing  and  Education  and  Training  cost  comparisons  have  been 
particularly  vulnerable  to  cancellation. 

By  time  period,  early  comparisons  were  generally  completed,  but 
comparisons  started  in  the  late  1980s  were  particularly  vulnerable  to 
cancellation. 

We  wondered  whether  Table  A.4's  civilian  billets  result  would  hold  for 
each  of  the  three  largest  services  separately.  Hence,  we  undertook  a 
supplemental  estimation  in  which  separate  SQRTCIVS  coefficients  were 
estimated  for  each  of  the  three  largest  services  (plus  a  fourth  for  the 
Marine  Corps  and  DoD  agencies).  Table  A.5  shows  the  results. 

Table  A.5  suggests  that  the  Navy  and,  particularly,  the  Army  have  had 
problems  with  cost  comparisons  that  affect  large  numbers  of  civilians 
being  disproportionately  canceled,  controlling  for  other  factors.  The  Air 
Force  does  not  appear  to  have  faced  similar  difficulties. 
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Table  A.3 

DoD  CAMIS  Completion  Probit  Variables 


Dependent  Variable 

1  if  comparison 
completed,  0  if 
canceled 

Independent  Variable^ 

Variable  Name 

Constant 

Square  root  of  number  of  military  slots  evaluated 

SQRTMILS 

Square  root  of  number  of  civilian  slots  evaluated 

SQRTCrVS 

Air  Combat  Command  installation  comparison 

ACC 

Air  Education  and  Training  Command  installation 

AETC 

comparison 

Air  Mobility  Command  installation  comparison 

AFAMC 

Air  Force  Materiel  Command  installation  comparison 

AFMC 

Space  Command  installation  comparison 

AFSPC 

Pacific  Air  Forces  installation  comparison 

PACAF 

Other  Air  Force  Command  installation  comparison 

AFOth 

Army  Materiel  Command  installation  comparison 

ArMC 

Forces  Command  installation  comparison 

FORS 

Training  and  Doctrine  Command  installation  comparison 

TRAD 

Other  Army  Command  installation  comparison 

ArOth 

Navy  comparison 

NAVY 

Marine  comparison 

USMC 

Defense  Logistics  Agency  comparison 

DLA 

Defense  Mapping  Agency  comparison 

DMA 

Automatic  Data  Processing 

ADP 

Base  Maintenance 

BASEMAIN 

Depot  Repair 

DEPOT 

Education  and  Training 

EDUCAT 

Health  Services 

HEALTH 

Intermediate  Maintenance 

IMAINT 

Manufacturing 

MANUF 

Other  Nonmanufacturing 

OTHERNON 

Research  &  Development 

R&D 

Real  Property  Maintenance 

RPM 

Social  Services 

SOCIAL 

FY  1980  or  earlier  comparison  start 

FY7980 

FY  1981-1982  comparison  start 

FY8182 

FY  1983-1984  comparison  start 

FY8384 

FY  1987-1988  comparison  start 

FY8788 

FY  1989-1990  comparison  start 

FY8990 

FY  1991  or  later  comparison  start 

FY9192 

®The  Defense  Commissary  Agency  is  the  omitted  command.  Installation  Services  is  the 
omitted  function,  and  comparisons  started  in  FY  1985  or  FY  1986  are  the  omitted  time  period 
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Table  A.4 

DoD  Cost  Comparison  Probit  on  Completion 


Dependent  Variable 

1  if  comparison  completed 

Observations 

3686 

Coefficient 

Standard  Error 

P  Value^ 

Intercept 

-0.0180 

0.2028 

0.9292 

SQRTMILS 

-0.0065 

0.0090 

0.4671 

SQRTCIVS 

-0.0386 

0.0069 

0.0001 

ACC 

0.1747 

0.1967 

0.3744 

AETC 

0.5987 

0.2048 

0.0035 

AFAMC 

0.0433 

0.2124 

0.8383 

AFMC 

0.1304 

0.1953 

0.5043 

AFSPC 

0.3581 

0.2618 

0.1713 

PACAF 

-0.5735 

0.2915 

0.0491 

AFOth 

-0.1007 

0.1905 

0.5971 

ArMC 

0.7741 

0.2299 

0.0008 

FORS 

-0.1790 

0.1877 

0.3402 

TRAD 

0.2959 

0.1951 

0.1294 

ArOth 

0.0982 

0.1934 

0.6116 

NAVY 

0.3455 

0.1778 

0.0520 

USMC 

-0.8391 

0.2190 

0.0001 

DLA 

-0.9155 

0.2574 

0.0004 

DMA 

-1.9254 

0.3361 

0.0001 

ADP 

-1.0989 

0.1028 

0.0001 

BASEMAIN 

0.0564 

0.2304 

0.8068 

DEPOT 

-0.8986 

0.2319 

0.0001 

EDUCAT 

-1.7525 

0.2353 

0.0001 

HEALTH 

-0.8083 

0.1530 

0.0001 

IMAINT 

0.0416 

0.1079 

0.6998 

MANUF 

-2.2639 

0.3646 

0.0001 

OTHERNON 

-0.4266 

0.0683 

0.0001 

R&D 

-0.2504 

0.2909 

0.3893 

RPM 

-0.0645 

0.0899 

0.4734 

SOCIAL 

-0.0331 

0.1097 

0.7626 

FY7980 

1.7065 

0.1181 

0.0001 

FY8182 

0.8830 

0.1008 

0.0001 

FY8384 

0.4952 

0.1042 

0.0001 

FY8788 

-0.6474 

0.1340 

0.0001 

FY8990 

-0.5033 

0.1496 

0.0008 

FY9192 

0.2786 

0.2441 

0.2537 

®The  P  value  column  in  this  case  shows  we  cannot  reject  a  null  that  the  ACC 
coefficient,  for  instance,  is  zero  at  the  95  percent  level,  but  we  can  reject  a  null  that  the 
SQRTCIVS  coefficient  is  zero  at  the  99.99  percent  confidence  level  (99.99  percent  =  1- 
0.0001).  This  ACC  estimate  suggests  we  cannot  reject  a  null  hypothesis  that  ACC's 
cancellation  probability  is  the  same  as  the  Defense  Commissary  Agency's,  the  omitted 
variable  in  this  category 


47 


Table  A.5 


DoD  Cost  Comparison  Probit  on  Completion  with  Service-Specific  SQRTCIVS 

Coefficients 


Dependent  Variable 

1  if  comparison  completed 

Observations 

3686 

Coefficient 

Standard  Error 

P  Value 

Intercept 

-0.2858 

0.2206 

0.1950 

SQRTMILS 

-0.0119 

0.0091 

0.1922 

Army  SQRTCIVS 

-0.0720 

0.0099 

0.0001 

Air  Force  SQRTCIVS 

0.0214 

0.0168 

0.2031 

Navy  SQRTCIVS 

-0.0291 

0.0122 

0.0166 

Other  SQRTCIVS 

0.0234 

0.0235 

0.3192 

ACC 

0.2852 

0.2193 

0.1783 

AETC 

0.6779 

0.2290 

0.0031 

AFAMC 

0.1530 

0.2343 

0.5137 

AFMC 

0.1296 

0.2271 

0.5681 

AFSPC 

0.4185 

0.2816 

0.1373 

PACAF 

-0.5105 

0.3103 

0.0999 

AFOth 

-0.0191 

0.2156 

0.9292 

ArMC 

1.3364 

0.2623 

0.0001 

FORS 

0.2759 

0.2151 

0.1996 

TRAD 

0.8366 

0.2281 

0.0002 

ArOth 

0.5522 

0.2195 

0.0119 

NAVY 

0.5760 

0.2044 

0.0048 

USMC 

-0.8467 

0.2214 

0.0001 

DLA 

-1.0490 

0.2670 

0.0001 

DMA 

-1.8339 

0.3388 

0.0001 

ADP 

-1.1138 

0.1031 

0.0001 

BASEMAIN 

-0.2520 

0.2375 

0.2886 

DEPOT 

-1.0190 

0.2325 

0.0001 

EDUCAT 

-1.7774 

0.2362 

0.0001 

HEALTH 

-0.8687 

0.1536 

0.0001 

IMAINT 

0.0664 

0.1097 

0.5451 

MANUF 

-2.1823 

0.3467 

0.0001 

OTHERNON 

-0.4292 

0.0686 

0.0001 

R&D 

-0.2808 

0.2880 

0.3296 

RPM 

-0.0960 

0.0910 

0.2914 

SOCIAL 

-0.0042 

0.1117 

0.9702 

FY7980 

1.7000 

0.1186 

0.0001 

FY8182 

0.8965 

0.1012 

0.0001 

FY8384 

0.4947 

0.1046 

0.0001 

FY8788 

-0.6587 

0.1344 

0.0001 

FY8990 

-0.4774 

0.1496 

0.0014 

FY9192 

0.2742 

0.2438 

0.2437 
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For  the  remainder  of  this  appendix,  we  focus  on  those  comparisons  that 
were  completed.  Completed  cost  comparisons  receive  a  dollar  value,  i.e., 
the  value  of  the  winning  contractor  or  government  employee  (labeled 
"MEO"  for  Most  Efficient  Organization)  bid.  CAMIS  dollar  totals  are 
then-year;  for  example,  a  cost  comparison  completed  in  1982  has  a  dollar 
total  in  1982  dollars.  To  compare  completed  cost  comparisons  over  time, 
we  standardized  all  cost  comparison  dollars  to  Fiscal  Year  (FY)  1994 
dollars.  Table  A.6  shows  the  multipliers  we  used  to  do  this. 

United  States  Department  of  Commerce  (1992)  and  the  Economic  Report  of 
the  President  (1996)  provide  first  and  second  calendar  quarter  aggregate 
price  indices  for  various  years  in  1987  and  1992  dollars,  respectively.  We 
assumed  the  average  of  a  calendar  year's  first  and  second  quarter  price 
indices  is  a  reasonable  price  index  for  the  corresponding  fiscal  year. 
(Fiscal  years  run  October-September.)  We  used  1991  data  to  link  the  two 
price  indices  (that  is,  establish  the  correct  relationship  between  1987  and 
1992  dollars).  Through  this  process,  we  computed  the  FY  1994  Multiplier 


Table  A.6 

FY  1994  Price  Index  Computation 


1987=100 

1992  =  100 

FY 

IstQ 

2nd  Q 

Average 

IstQ 

2ndQ 

Average 

FY1994 

Multiplier 

1977 

54.3 

55.4 

54.8 

2.29024 

1978 

58.2 

59.7 

59.0 

2.13095 

1979 

63.5 

64.8 

64.2 

1.95821 

1980 

69.2 

70.8 

70.0 

1.79456 

1981 

76.5 

77.9 

77.2 

1.62720 

1982 

82.3 

83.4 

82.8 

1.51623 

1983 

86.0 

86.6 

86.3 

1.45561 

1984 

89.7 

90.6 

90.2 

1.39345 

1985 

93.3 

94.0 

93.6 

1.34137 

1986 

96.0 

96.5 

96.2 

1.30514 

1987 

98.8 

99.5 

99.2 

1.26696 

1988 

102.1 

103.2 

102.6 

1.22377 

1989 

106.9 

108.0 

107.4 

1.16910 

1990 

111.1 

112.3 

111.7 

1.12461 

1991 

115.9 

116.8 

116.4 

96.3 

97.0 

96.6 

1.07967 

1992 

99.1 

99.8 

99.4 

1.04927 

1993 

101.8 

102.4 

102.1 

1.02204 

1994 

104.1 

104.6 

104.4 

1.00000 

SOURCES:  United  States  Department  of  Commerce  (1992)  and  Economic  Report  of  the  President 
(1996). 
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column  shown  in  Table  A.6.  We  multiplied  all  dollar  totals  in  CAMIS  by 
the  appropriate  multiplier  for  the  fiscal  year  in  which  a  cost  comparison 
was  completed.  All  dollar  totals  used  in  this  analysis  are  in  FY 1994 
dollars.  Clearly,  for  early  cost  comparisons  particularly,  the  FY  1994 
dollar  value  is  considerably  greater  than  the  then-year  nominal  dollar 
value  actually  provided  in  CAMIS. 

The  CAMIS  system  contains  a  variety  of  date  stamps.  These  stamps 
include  the  date  a  cost  comparison  was  announced,  the  date  of  an  initial 
decision,  the  date  of  the  final  decision,  and  the  date  the  contractor  or  MEO 
started. 

The  contract/MEO  start  date  is  the  date  the  winning  contractor  or  in- 
house  employees  began  performing  the  function.  However,  this  field  is 
often  missing  in  the  data  set.  When  the  contract/MEO  start  date  is 
provided,  it  often  precedes  the  final  decision  date.  The  CAMIS  records 
indicate  the  winning  contractor  or  MEO  often  began  work  prior  to  the 
final  decision.  Hence,  we  decided  to  focus  on  the  announcement  date  and 
the  initial  decision  date  as  a  reasonable  approximation  for  the  length  of  the 
cost  comparison  process. 

For  the  duration  analysis,  we  focused  on  completed  cost  comparisons 
only.  That  is,  a  decision  was  made  that  either  government  employees  or  a 
specific  contractor  would  perform  the  specified  function.  Also,  we  deleted 
records  lacking  a  contract  dollar  value  or  duration. 

In  total,  we  were  left  with  2134  completed  cost  comparisons.  Table  A.7 
presents  some  basic  statistics  on  our  data.  Days  in  our  analysis  are 
calendar  days,  not  business  days.  Table  A.7  and  all  subsequent  tables 
cover  only  the  2134  completions  that  were  not  omitted  because  of  the 
aforementioned  data  problems.  Annualized  dollars  present  the  FY  1994 
per-year  value  of  the  winning  completed  cost  comparison  bid — for 
exmple,  the  MEO  value  if  government-employed  civilians  won  the 
competition.  Civilian  slots  is  the  number  of  government-employed 

Table  A.7 


DoD  CAMIS  Completion  Siimmary 


Statistic 

Min 

5% 

25% 

Med 

Mean 

75% 

95% 

Max 

Time  to  initial  decision  (days) 

10 

189 

452 

664 

809 

1041 

1902 

3444 

Annualized  FY94  dollars  (000s) 

7 

59 

192 

474 

1515 

1176 

6076 

83350 

Civilian  slots  compared 

0 

1 

5 

12 

31 

28 

125 

1057 

Military  slots  compared 

0 

0 

0 

0 

9 

1 

26 

1016 
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civilian  slots  evaluated  in  completed  cost  comparisons;  military  slots  is  the 
number  of  military  slots  evaluated. 

Each  CAMIS  record  also  includes  a  Department  of  Defense  Fimctional 
Activity  Code  (DoDFAC),  a  categorization  of  the  type  of  function 
competed.  Table  A.8  presents  information  about  the  completed  cost 
comparisons  by  DoDFAC.  As  noted  above,  some  CAMIS  records  listed 
multiple  functions;  we  used  the  first  listed  function. 

In  real  dollars.  Installation  Services  is  the  high  dollar  category  in  these 
data.  There  have  been  nearly  as  many  completed  comparisons  in  Other 
Nonmanufacturingi  as  in  Installation  Services,  but  Installation  Services' 
average  completed  cost  comparison  has  been  more  than  70  percent  larger. 

It  is  also  interesting  to  note  that  whereas  government  employees  have 
won  about  half  of  the  A-76  cost  comparisons,  their  success  rate  varies 
considerably  across  the  categories,  e.g.,  80  percent  in  Health  Services 
versus  21  percent  in  Social  Services. 

Table  A.9  shows  that  FY  1983  had  the  largest  number  of  A-76  cost 
comparison  completions  and  FY  1980  the  highest  real  dollar  value.  There 
has  been  a  distinct  decline  in  A-76  activity  in  recent  years.  This  decline 


Table  A.8 

DoD  Completed  Cost  Comparisons  by  DoD  Functional  Activity  Code 


DoDFAC 

Total 

Annualized 
Dollars  (000s) 

Average 
Value  (000s) 

Government 

Employees 

Won 

Automatic  Data  Processing 

95 

66,339 

698 

55 

Base  Maintenance 

28 

191,957 

6,856 

16 

Depot  Repair 

9 

26,877 

2,986 

9 

Education  and  Training 

7 

35,088 

5,013 

3 

Health  Services 

35 

25,351 

724 

28 

Installation  Services 

655 

1,062,362 

1,622 

349 

Intermediate  Maintenance 

163 

573,023 

3,515 

66 

Manufacturing 

2 

63,168 

31,584 

0 

Other  Nonmanufacturing 

588 

543,717 

925 

334 

Research  &  Development 

12 

75,629 

6,302 

3 

Real  Property  Maintenance 

316 

456,712 

1,445 

144 

Social  Services 

224 

113,341 

506 

48 

All  completions 

2134 

3,233,565 

1,515 

1055 

^The  "Other  Nonmanufacturing"  DoDFAC  is  largely  composed  of  logistics-related 
fimctions  such  as  storage  and  warehousing,  transportation  services,  property  disposal. 
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Table  A.9 


DoD  Completed  Cost  Comparisons  by  Fiscal  Year 
of  Initial  Decision 


FY 

Annualized 

Average 

Dollars 

Annualized 

Year 

Total 

(OOOs) 

Dollars  (OOOs) 

1978 

1 

1,536 

1,536 

1979 

114 

152,934 

1,342 

1980 

205 

557,496 

2,719 

1981 

186 

369,334 

1,986 

1982 

292 

428,160 

1,466 

1983 

370 

390,584 

1,056 

1984 

171 

161,025 

942 

1985 

192 

263,502 

1,372 

1986 

184 

203,770 

1,107 

1987 

127 

239,844 

1,889 

1988 

109 

177,548 

1,629 

1989 

70 

156,242 

2,232 

1990 

29 

35,502 

1,224 

1991 

66 

67,945 

1,029 

1992 

9 

11,369 

1,263 

1993 

2 

7,712 

3,856 

1994 

7 

9,063 

1,295 

Total 

2134 

3,233,565 

1,515 

probably  results,  at  least  in  part,  from  the  FY 1989  National  Defense 
Authorization  Act  directive  that  installation  commanders  have  the  sole 
authority  to  determine  which  functions  to  cost  compare  or  direct  convert 
(10  U.S.C.  2468).  (This  stipulation  expired  September  30, 1995.) 

Installation  commanders  on  comparatively  short  tours  are  often  reluctant 
to  undertake  painful  A-76  cost  comparisons,  even  if  such  cost  comparisons 
have  long-term  benefits.^  Further,  the  FY  1993  DoD  Authorization  Act 
imposed  a  DoD-wide  moratorium  on  awarding  contracts  resulting  from 
cost  comparisons;  it  was  removed  effective  April  1, 1994. 

Table  A.IO  presents  the  number  and  annualized  dollar  value  of  A-76  cost 
comparisons  won  by  government  employees  and  contractors.  Overall, 
government  employees  won  49  percent  of  the  A-76  cost  comparisons 
covering  43  percent  of  the  competed  annualized  FY  1994  dollar  value. 


^Bolten,  Halliday,  and  Keating  (1996)  discuss  this  problem  in  the  Army  and  present  some 
possible  solutions  (e.g.,  longer  military  tours  or  greater  civilian  control  of  installations). 
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Table  A.10 

Completed  Cost  Comparison  Outcomes  by  Fiscal  Year  of  Initial  Decision 


FY 

Year 

Government 
Employees  Won 

Annualized 
Dollars  (000s) 

Contractors 

Won 

Annualized 
Dollars  (000s) 

1978 

0 

0 

1 

1,536 

1979 

29 

55,611 

85 

97,324 

1980 

82 

199,000 

123 

358,497 

1981 

79 

111,819 

107 

257,515 

1982 

154 

185,009 

138 

243,151 

1983 

206 

202,910 

164 

187,674 

1984 

101 

104,804 

70 

56,221 

1985 

109 

120,763 

83 

142,739 

1986 

86 

100,985 

98 

102,785 

1987 

62 

99,978 

65 

139,866 

1988 

48 

79,316 

61 

98,233 

1989 

41 

75,085 

29 

81,158 

1990 

15 

19,291 

14 

16,211 

1991 

35 

27,505 

31 

40,440 

1992 

5 

3,706 

4 

7,663 

1993 

1 

355 

1 

7,356 

1994 

2 

2,455 

5 

6,608 

Total 

1055 

1,388,591 

1079 

1,844,974 

Table  A.ll  tallies  completed  cost  comparisons  and  annualized  FY  1994 
dollar  value  by  service  and  year.  The  Navy  completed  the  most 
comparisons,  but  the  Army's  comparisons  encompassed  the  greatest 
value.  Notice  that  for  every  service,  the  standard  deviation  of  completed 
cost  comparison  size  is  considerably  greater  than  the  mean  completed  cost 
comparison  size.  Completed  cost  comparisons  vary  greatly  in  size  within 
services. 

We  undertook  another  probit  estimation  to  analyze  when  contractors,  as 
opposed  to  government  employees,  won  cost  comparisons.  Table  A.12 
presents  our  independent  variables.  Our  independent  variables  are  the 
same  as  in  Table  A.3,  except  DEPOT  and  MAfWF  are  no  longer 
considered  (because  of  a  lack  of  variance  in  their  cost  comparison  winners) 
and  we  added  dummy  variables  for  sealed  bid,  negotiated,  and 
imrestricted  cost  comparisons.  (These  three  new  dummy  variable 
categories  could  not  be  used  in  Table  A.4's  estimation  because  these  fields 
were  frequently  left  empty  in  cost  comparisons  that  were  canceled.) 

Table  A.13  presents  our  winner  probit  estimates. 
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Table  A.ll 


Completed  Cost  Comparisons  by  Service  and  Fiscal  Year  of  Initial  Decision 


FY 

Year 

AF 

No. 

Annual 

Dollars  Army 
(000s)  No. 

Annual 

Dollars 

(000s) 

Navy 

No. 

Annual 

Dollars 

(000s) 

USMC 

No. 

Annual 

Dollars 

(000s) 

DoD  Annual 
Agnc  Dollars 
No.  (000s) 

1978 

0 

0 

1 

1,536 

0 

0 

0 

0 

0 

0 

1979 

55 

113,002 

14 

16,042 

41 

20,701 

4 

3,190 

0 

0 

1980 

105 

168,201 

67 

374,044 

28 

12,857 

5 

2,393 

0 

0 

1981 

88 

203,091 

79 

152,668 

15 

11,351 

4 

2,223 

0 

0 

1982 

61 

48,804 

59 

207,183 

167 

162,983 

2 

4,213 

3 

4,978 

1983 

83 

39,474 

60 

135,022 

216 

206,570 

8 

7,232 

3 

2,284 

1984 

42 

30,948 

25 

43,500 

98 

82,688 

4 

1,981 

2 

1,908 

1985 

65 

44,995 

41 

60,146 

75 

113,158 

7 

9,418 

4 

35,785 

1986 

57 

34,976 

73 

95,188 

50 

64,925 

3 

8,014 

1 

668 

1987 

64 

50,451 

14 

58,430 

42 

121,849 

3 

8,430 

4 

683 

1988 

27 

72,159 

39 

53,835 

25 

46,722 

0 

0 

18 

4,831 

1989 

33 

90,862 

12 

17,821 

17 

44,263 

2 

1,131 

6 

2,165 

1990 

17 

24,550 

7 

3,600 

0 

0 

2 

5,788 

3 

1,564 

1991 

50 

52,229 

6 

10,222 

1 

2,820 

0 

0 

9 

2,674 

1992 

6 

4,919 

2 

6,006 

1 

443 

0 

0 

0 

0 

1993 

1 

7,356 

1 

355 

0 

0 

0 

0 

0 

0 

1994 

6 

8,483 

1 

579 

0 

0 

0 

0 

0 

0 

Tot 

760 

994,501 

501  1, 

,236,177 

776 

891,333 

44 

54,014 

53 

57,540 

Average 

1,309 

2,467 

1,149 

1,228 

1,086 

Std  Dev 

3,304 

6,683 

2,787 

1,710 

2,425 

Table  A.13  suggests,  controlling  for  other  factors,  that  contractors  were 
increasingly  likely  to  win  cost  comparisons  the  smaller  the  fraction  of 
evaluated  slots  originally  held  by  government-employed  civilians. 

Among  the  commands,  contractors  had  the  greatest  relative  success 
winning  Air  Force  Space  Command  installation  cost  comparisons  and  the 
least  success  in  DoD  agency  comparisons  (the  omitted  command /service). 

By  function,  controlling  for  other  factors,  the  estimation  suggests  that 
contractors  had  the  greatest  success  in  Social  Services  cost  comparisons 
and  the  least  success  in  Health  Services  cost  comparisons.  However,  as 
noted  in  Table  A.8,  the  data  also  included  two  Manufacturing  cost 
comparisons,  both  won  by  contractors,  and  nine  Depot  Repair  cost 
comparisons,  all  won  by  government  employees.  Given  the  lack  of 
variance  in  these  functions'  outcomes,  we  had  to  omit  them  from  this 
probit  estimation. 
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Table  A.12 

DoD  CAMIS  Winner  Probit  Variables 


Dependent  Variable 

1  if  contractor  won, 

0  if  government 
employees  won 

Independent  Variable^ 

Variable  Name 

Constant 

Square  root  of  annualized  FY  1994  $  value  of  contract 

SQRTANMO 

Fraction  of  evaluated  slots  held  by  government-employed 

CIVSLFRAC 

civilians 

Air  Combat  Command  installation  comparison 

ACC 

Air  Education  and  Training  Command  installation  comparison 

AETC 

Air  Mobility  Command  installation  comparison 

AFAMC 

Air  Force  Materiel  Command  installation  comparison 

AFMC 

Space  Command  installation  comparison 

AFSPC 

Pacific  Air  Forces  installation  comparison 

PACAF 

Other  Air  Force  Command  installation  comparison 

AFOth 

Army  Materiel  Command  installation  comparison 

ArMC 

Forces  Command  installation  comparison 

FORS 

Training  and  Doctrine  Command  installation  comparison 

TRAD 

Other  Army  Command  installation  comparison 

ArOth 

Navy  comparison 

NAVY 

Marine  comparison 

USMC 

Automatic  Data  Processing 

ADP 

Base  Maintenance 

BASEMAIN 

Education  and  Training 

EDUCAT 

Health  Services 

HEALTH 

Intermediate  Maintenance 

IMAINT 

Other  Nonmanufacturing 

OTHERNON 

Research  &  Development 

R&D 

Real  Property  Maintenance 

RPM 

Social  Services 

SOCIAL 

FY  1979-1980  comparison  completion 

FY7980 

FY  1981-1982  comparison  completion 

FY8182 

FY  1983-1984  comparison  completion 

FY8384 

FY  1987-1988  comparison  completion 

FY8788 

FY  1989-1989  comparison  completion 

FY8990 

FY  1991-1992  comparison  completion 

FY9192 

FY  1993-1994  comparison  completion 

FY9394 

Sealed  Bid 

SEALED 

Negotiated 

NEGOTIATE 

Unrestricted  cost  comparison 

UNRESTRICTED 

^DoD  agencies'  comparisons  is  the  omitted  command.  We  combine  DeCA  (39  completions) 
and  DLA  (14  completions)  in  this  category.  Meanwhile,  the  Defense  Mapping  Agency  had  three 
completions,  but  they  were  omitted  from  this  analysis  because  of  lack  of  dollar  value  or  duration  in 
the  CAMIS  data.  Installation  Services  is  the  omitted  function;  comparisons  completed  in  FY  1985  or 
FY  1986  is  the  omitted  time  period;  other  types  of  contracts,  including  no  type  specified,  is  the 
omitted  contract  type;  and  other  types  of  initiatives  (e.g.,  restricted  to  small  or  disadvantaged 
businesses)  is  the  omitted  initiative  type. 
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Table  A.13 

DoD  Completed  Cost  Comparisons  Probit  on  Winner 


Dependent  Variable 

1  if  comparison 

won  by  contractor 

Observations 

2098 

Coefficient 

Standard  Error 

P  Value 

Intercept 

0.1382 

0.5765 

0.8106 

SQRTANMO 

0.0011 

0.0013 

0.3787 

CIVSLFRAC 

-0.5851 

0.1237 

0.0001 

ACC 

0.5223 

0.2401 

0.0296 

AETC 

0.5934 

0.2413 

0.0139 

AFAMC 

0.4449 

0.2556 

0.0818 

AFMC 

0.6744 

0.2422 

0.0054 

AFSPC 

0.7203 

0.2936 

0.0142 

PACAF 

0.3498 

0.3990 

0.3806 

AFOth 

0.4798 

0.2378 

0.0436 

ArMC 

0.0576 

0.2594 

0.8243 

FORS 

0.3213 

0.2409 

0.1824 

TRAD 

0.2719 

0.2412 

0.2596 

ArOth 

0.3141 

0.2488 

0.2067 

NAVY 

0.2760 

0.2211 

0.2120 

USMC 

0.1985 

0.2915 

0.4959 

ADP 

-0.0374 

0.1457 

0.7975 

BASEMAIN 

-0.1359 

0.2583 

0.5990 

EDUCAT 

0.1431 

0.4952 

0.7726 

HEALTH 

-0.9561 

0.2646 

0.0003 

IMAINT 

0.2350 

0.1205 

0.0511 

OTHERNON 

-0.1089 

0.0810 

0.1786 

R&D 

0.6466 

0.4734 

0.1719 

RPM 

0.0684 

0.0990 

0.4895 

SOCIAL 

1.1147 

0.1230 

0.0001 

FY7980 

0.3292 

0.1045 

0.0016 

FY8182 

0.0661 

0.0941 

0.4828 

FY8384 

-0.1392 

0.0911 

0.1265 

FY8788 

0.8415 

0.1114 

0.4502 

FY8990 

-0.3038 

0.1557 

0.0510 

FY9192 

-0.4453 

0.1872 

0.0174 

FY9394 

0.3059 

0.5179 

0.5547 

SEALED 

-0.1072 

0.5161 

0.8354 

NEGOHATE 

-0.1958 

0.5191 

0.7061 

UNRESTRICTED 

0.0977 

0.0677 

0.1488 
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A  key  question  we  wished  to  evaluate  was  why  the  process  (as  shown  by 
these  data)  has  been  so  lengthy  and  so  highly  variable  (see  Table  A.7).  As 
a  first  step  toward  answering  this  question,  we  regressed  the  square  root 
of  observed  total  duration  from  the  Announcement  to  the  Initial  Decision 
on  the  same  explanatory  variables  shown  in  Table  A.12  and  used  in  Table 
A.13,  except  that  Depot  Repair  and  Manufacturing  were  also  considered, 
with  Manufacturing  the  omitted  function. 

Table  A.14  presents  our  regression  results.  Table  A.15  shows  the 
covariance-generated  square  root  means  and  Tukey-Kramer  adjusted 
standard  error  estimates  for  the  square  root  of  typical  values  of  some  of 
the  dummy  variable  categories.  Pearce  (1982)  describes  analysis  of 
covariance.  Least-squares  square  root  means,  as  shown  in  Table  A.15,  are 
estimates  of  the  square  root  means  that  would  be  expected  had  other 
dummy  variables  been  balanced  in  their  frequency.  They  do  not 
necessarily  correspond  in  magnitude  to  sample  square  root  means  in  the 
data.  See  SAS  Institute  (1990b),  Volume  2,  pages  908  and  948.  Tukey- 
Kramer  standard  errors,  meanwhile,  are  designed  for  pairwise 
comparisons.  Two  square  root  means  in  Table  A.15  are  significantly 
different  if  one  adjusted  standard  error  below  the  larger  is  larger  than  one 
adjusted  standard  error  above  the  smaller.  See  Miller  (1985). 

Table  A.14  shows  that  the  independent  variables  explain  only  about  50 
percent  of  the  variance  in  the  data.  The  cost  comparison  process  is  not 
only  lengthy,  but  subject  to  variation  not  captured  by  the  explanatory 
variables  we  have  employed. 

Among  the  commands  and  services,  the  Army's  Forces  and  Materiel 
commands'  installations  are  statistically  significantly  slower  at  the  95 
percent  confidence  level  than  the  defense  agencies'  zero  baseline.  None  of 
the  function  dummy  variables  is  statistically  significant. 

Tables  A.14  and  A.15  suggest  that  the  cost  comparison  process  slowed 
considerably  in  the  late  1980s.  However,  this  finding  may  be  spurious. 
Cost  comparisons  started  in  earnest  only  in  the  late  1970s  so,  by  definition, 
any  comparisons  completed  in  the  late  1970s  or  early  1980s  had  to  be 
rapid.  In  the  mid-  and  late-1980s,  many  lengthy  comparisons  started  in 
the  late  1970s  and  early  1980s  were  completed. 

There  is  some  suggestion  that  the  cost  comparison  process  has  sped  up  in 
recent  years.  Comparisons  completed  in  FY  1989-1990  typically  took 
some  678  days  longer  than  comparisons  completed  FY  1993  and  later, 
whereas  comparisons  completed  in  FY  1991-1992  typically  took  an 
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Table  A.14 


DoD  Completed  Cost  Comparisons  Square  Root  of  Time  to 
Initial  Decision  Linear  Regression 


Dependent  Variable 

Square  root  of  total  duration 

R-Squared^ 

0.4910 

Observations 

2109 

Analysis  of  variance 

df 

Sum  of  squares 

pb:  55.511 

Regression 

36 

81859.1234 

Residual 

2072 

84874.2580 

Significance  F: 

Total 

2108 

166733.3814 

0.0001 

Coefficient 

Standard  Error 

P  Value 

Intercept 

28.0467 

2.8928 

0.0001 

SQRTANMO 

0.0111 

0.0062 

0.0708 

CIVSLFRAC 

-1.2709 

0.6042 

0.0355 

ACC 

0.1277 

1.1126 

0.9086 

AETC 

-1.5163 

1.1257 

0.1781 

AFAMC 

0.6091 

1.1932 

0.6098 

AFMC 

--0.2770 

1.1229 

0.8052 

AFSPC 

-1.1064 

1.3864 

0.4249 

PACAF 

0.3711 

1.9645 

0.8502 

AFOth 

0.1480 

1.1008 

0.8930 

ArMC 

2.7682 

1.2079 

0.0220 

FORS 

3.2412 

1.1213 

0.0039 

TRAD 

2.1544 

1.1219 

0.0549 

ArOth 

2.1192 

1.1631 

0.0686 

NAVY 

-0.5644 

1.1045 

0.5780 

USMC 

2.2408 

1.3916 

0.1075 

ADP 

2.9364 

0.7256 

0.0001 

BASEMAIN 

2.4015 

1.3008 

0.0650 

DEPOT 

-0.1738 

2.2081 

0.9373 

EDUCAT 

-5.2659 

2.4495 

0.0317 

HEALTH 

2.3751 

1.1679 

0.0421 

IMAINT 

-1.3086 

0.5941 

0.0277 

MANUF 

-3.5429 

4.6240 

0.4437 

OTHERNON 

0.1794 

0.4042 

0.6573 

R&D 

-1.0103 

2.0079 

0.6149 

RPM 

1.1164 

0.4959 

0.0245 

SOCIAL 

0.1896 

0.5527 

0.7317 

FY7980 

-10.0411 

0.5114 

0.0001 

FY8182 

-3.7264 

0.4632 

0.0001 

FY8384 

-2.8512 

0.4477 

0.0001 

FY8788 

8.3766 

0.5493 

0.0001 

FY8990 

11.8051 

0.7627 

0.0001 
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Table  A. 14 — continued 


Coefficient 

Standard  Error 

P  Value 

FY9192 

4.9607 

0.9414 

0.0001 

FY9394 

2.2167 

2.4973 

0.3748 

SEALED 

0.1771 

2.6338 

0.9464 

NEGOTIATE 

1.5301 

2.6478 

•  0.5634 

UNRESTRICTED 

0.1059 

0.3358 

0.7526 

^R-squared  is  a  measure  of  regression  goodness-of-fit.  If  the  independent  variables 
explained  all  variance  in  the  dependent  variable,  the  R-squared  would  be  1.  In  this  case, 
the  independent  variables  explain  about  50  percent  of  the  variance  in  the  dependent 
variable. 

^The  regression  F  statistic  is  a  test  of  the  null  hypothesis  that  all  the  regression 
coefficients,  except  the  intercept,  are  actually  zero.  In  this  case,  the  F  statistic  of  55.511  is 
sufficiently  large  that  we  can  reject  the  null  that  all  the  non-intercept  coefficients  are  zero 
at  an  extremely  high  (99.99%  =  1  -  0.0001)  confidence  level. 


estimated  175  days  longer  than  comparisons  completed  in  FY 1993  and 
later.  A  possible  explanation  may  lie  in  the  policy  (first  promulgated  in 
Section  8020  of  the  FY  1991  Appropriations  Act)  of  canceling  single 
function  cost  comparisons  after  two  years  and  multifunction  cost 
comparisons  after  four  years  if  they  have  not  yet  reached  bid  opening. 
Perhaps  the  apparent  speed  increase  results  from  data  censoring  caused 
by  more  cost  comparisons  being  canceled. 

However,  only  nine  cost  comparisons  were  completed  in  FY  1993  and 
later.  Maybe  personnel  simply  had  a  lower  A-76  workload  in  recent 
periods.  The  apparent  speed  increase  may  not  persist  if  the  comparison 
workload  increases. 

As  a  complement  to  Table  A.14's  analysis,  we  ran  a  probit  estimation.  The 
dependent  variable,  LONG,  was  coded  "1"  if  a  completed  cost  comparison 
ran  over  a  calendar  year  or  if  a  cost  comparison  was  ongoing  for  over  a 
year.  We  wished  to  omit  ongoing  cost  comparisons  that  had  yet  to  run  for 
a  year,  so  we  removed  all  FY  1994-started  cost  comparisons.  Unlike  Table 
A.14,  the  time  dummy  variables  in  this  estimation  cover  starting  year,  not 
year  of  completion. 

This  estimation  suggests  that  cost  comparisons  that  evaluated  more  slots 
were  more  likely  to  run  over  a  year,  other  things  being  equal. 

The  Air  Force,  Navy,  and  Marine  Corps  coefficients  are  negative, 
suggesting  that  the  omitted  category — ^DoD  agencies  and  the  Army — 
tended  to  have  lengthy  cost  comparisons.  Indeed,  every  Army  cost 
comparison  in  the  data  ran  over  a  year,  so  an  Army  coefficient  per  se  was 
not  estimable.  For  the  same  reason,  we  could  not  run  this  estimation  with 
the  Air  Force  broken  into  commands. 
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Table  A.15 


DoD  Completed  Cost  Comparisons  Square  Root  of 
Time  to  Initial  Decision,  Least-Squares  Means  and 
Tukey-Kramer  Adjusted  Standard  Errors 


Variable  Name 

Mean 

Tukey-Kramer  Adjusted 
Std  Error 

ACC 

29.1713 

1.1918 

AETC 

27.5260 

1.2047 

AFAMC 

29.6536 

1.2836 

AFMC 

28.7655 

1.2025 

AFSPC 

27.9364 

1.4662 

PACAF 

29.4166 

2.0344 

AFOth 

29.1913 

1.1877 

ArMC 

31.8084 

1.2942 

FORS 

32.2223 

1.1934 

TRAD 

31.2138 

1.2276 

ArOth 

31.1622 

1.2205 

NAVY 

28.4777 

1.0866 

USMC 

31.2844 

1.4615 

DoD  Agencies 

29.0473 

1.4134 

ADP 

32.8897 

1.1873 

BASEMAIN 

32.3545 

1.5605 

DEPOT 

29.7713 

2.3854 

EDUCAT 

24.6960 

2.6283 

HEALTH 

32.3306 

1.4390 

INSTALL 

29.9501 

0.9851 

IMAINT 

28.6433 

1.0852 

MANUF 

26.4047 

4.7234 

OTHERNON 

30.1325 

1.0019 

R&D 

28.9435 

2.1993 

RPM 

31.0684 

1.0332 

SOCIAL 

30.1384 

1.0658 

FY7980 

18.3949 

1.0848 

FY8182 

24.7012 

1.0810 

FY8384 

25.5854 

1.0890 

FY8586 

28.4371 

1.0902 

FY8788 

36.8113 

1.1220 

FY8990 

40.2271 

1.2234 

FY9192 

33.3918 

1.3583 

FY9394 

30.6667 

2.6795 
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Table  A.16 

DoD  Cost  Comparison  Probit  on  Duration  Over  One  Year 


Dependent  Variable 

1  if  comparison  lasted  over  a  year 

Observations 

2331 

Coefficient 

Standard  Error 

P  Value 

Intercept 

1.3455 

0.2305 

0.0001 

SQRTMILS 

0.0658 

0.0215 

0.0022 

SQRTCIVS 

0.0341 

0.0129 

0.0080 

AIRFORCE 

-0.7498 

0.1097 

0.0001 

NAVY 

-0.9369 

0.1083 

0.0001 

USMC 

-1.1237 

0.2288 

0.0001 

ADP 

0.5044 

0.1582 

0.0027 

BASEMAIN 

0.7092 

0.4835 

0.1424 

DEPOT 

0.5455 

0.5395 

0.3120 

EDUCAT 

-0.3048 

0.4946 

0.5377 

HEALTH 

0.8354 

0.3846 

0.0298 

IMAINT 

0.1668 

0.1459 

0.2530 

MANUF 

-0.8311 

0.7800 

0.2866 

R&D 

0.3319 

0.4485 

0.4593 

RPM 

0.5565 

0.1183 

0.0001 

SOCIAL 

0.6923 

0.1469 

0.0001 

FY7980 

-0.4456 

0.2092 

0.0332 

FY8182 

0.2857 

0.2056 

0.1646 

FY8384 

0.0714 

0.2165 

0.7415 

FY8788 

0.4402 

0.3567 

0.2172 

FY8990 

-0.7882 

0.2653 

0.0030 

FY9193 

-0.3149 

0.3575 

0.3784 

Measured  by  starting  year,  cost  comparisons  started  in  FY  1979  and  1980 
and  again  in  FY  1989  and  1990  were  statistically  significantly  more  likely 
to  rim  less  than  a  year  than  FY  1985  and  1986-started  cost  comparisons, 
the  omitted  category.  The  FY  1989  and  1990  result  combined  with  the  FY 
1991-1993  negative  point  estimate  provide  some  support  to  the  notion  that 
more  recently  started  cost  comparisons  have  run  somewhat  more  quickly. 
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